23 research outputs found
Tracking interacting targets in multi-modal sensors
PhDObject tracking is one of the fundamental tasks in various applications such as surveillance,
sports, video conferencing and activity recognition. Factors such as occlusions,
illumination changes and limited field of observance of the sensor make tracking a challenging
task. To overcome these challenges the focus of this thesis is on using multiple
modalities such as audio and video for multi-target, multi-modal tracking. Particularly,
this thesis presents contributions to four related research topics, namely, pre-processing of
input signals to reduce noise, multi-modal tracking, simultaneous detection and tracking,
and interaction recognition.
To improve the performance of detection algorithms, especially in the presence
of noise, this thesis investigate filtering of the input data through spatio-temporal feature
analysis as well as through frequency band analysis. The pre-processed data from multiple
modalities is then fused within Particle filtering (PF). To further minimise the discrepancy
between the real and the estimated positions, we propose a strategy that associates the
hypotheses and the measurements with a real target, using a Weighted Probabilistic Data
Association (WPDA). Since the filtering involved in the detection process reduces the
available information and is inapplicable on low signal-to-noise ratio data, we investigate
simultaneous detection and tracking approaches and propose a multi-target track-beforedetect
Particle filtering (MT-TBD-PF). The proposed MT-TBD-PF algorithm bypasses
the detection step and performs tracking in the raw signal. Finally, we apply the proposed
multi-modal tracking to recognise interactions between targets in regions within, as well
as outside the cameras’ fields of view.
The efficiency of the proposed approaches are demonstrated on large uni-modal,
multi-modal and multi-sensor scenarios from real world detections, tracking and event
recognition datasets and through participation in evaluation campaigns
Point cloud segmentation using hierarchical tree for architectural models
Recent developments in the 3D scanning technologies have made the generation
of highly accurate 3D point clouds relatively easy but the segmentation of
these point clouds remains a challenging area. A number of techniques have set
precedent of either planar or primitive based segmentation in literature. In
this work, we present a novel and an effective primitive based point cloud
segmentation algorithm. The primary focus, i.e. the main technical contribution
of our method is a hierarchical tree which iteratively divides the point cloud
into segments. This tree uses an exclusive energy function and a 3D
convolutional neural network, HollowNets to classify the segments. We test the
efficacy of our proposed approach using both real and synthetic data obtaining
an accuracy greater than 90% for domes and minarets.Comment: 9 pages. 10 figures. Submitted in EuroGraphics 201
Multi-modal particle filtering tracking using appearance, motion and audio likelihoods
ABSTRACT We propose a multi-modal object tracking algorithm that combines appearance, motion and audio information in a particle filter. The proposed tracker fuses at the likelihood level the audio-visual observations captured with a video camera coupled with two microphones. Two video likelihoods are computed that are based on a 3D color histogram appearance model and on a color change detection, whereas an audio likelihood provides information about the direction of arrival of a target. The direction of arrival is computed based on a multi-band generalized cross-correlation function enhanced with a noise suppression and reverberation filtering that uses the precedence effect. We evaluate the tracker on single and multi-modality tracking and quantify the performance improvement introduced by integrating audio and visual information in the tracking process
Analyzing the Knock-on Impacts of 2022 Floods on Rabi 2023 Using Remote Sensing and Field Surveys
While the world's attention is focused on immediate relief and rescue
operations for the affectees of the current floods in Pakistan, knock-on
effects are expected to play further havoc with the country's economy and food
security in the coming months. Significant crop yield losses had already
occurred for Winter (Rabi) 2021-22 due to a heatwave earlier in the year and
estimates for the Summer (Kharif) 2022 crop damage due to flood inundation have
already been determined to be very high. With the next sowing season already
upon the flood affectees, there is a big question mark over the resumption of
agricultural activity in disaster-struck districts. This study is aimed at
analyzing the range of influences of the 2022 floods on the upcoming winter
(Rabi) crop. Satellite-based remote sensing data, state-of-the-art Earth system
models, and field observations will be leveraged to estimate the impacts of the
flood on the resumption of agricultural activity in the most impacted districts
of Southern Punjab, Sindh, and Baluchistan. The field surveys are conducted
during multiple visits to the study area to maximize the monitoring of
on-ground conditions and provide a larger validation dataset for the
satellite-based inundation and crop classification maps. The project leverages
on the expertise and previous experiences of the LUMS team in performing
satellite-based land/crop classification, estimation of soil moisture levels
for irrigation activity, and determining changes in land-use patterns for
detecting key agricultural activities. Delays in the sowing of the winter crop
and its effects on crop-yield were analyzed through this study